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Abstract 

Background: Assessment of hand kinennatics is innportant when evaluating hand functioning. Major drawbacks of 
current sensing glove systems are lack of rotational observability in particular directions, labour intensive calibration 
methods which are sensitive to wear and lack of an absolute hand orientation estimate. 

Methods: We propose an ambulatory system using inertial sensors that can be placed on the hand, fingers and 
thumb. It allows a full 3D reconstruction of all finger and thumb joints as well as the absolute orientation of the hand. 
The system was experimentally evaluated for the static accuracy, dynamic range and repeatability. 

Results: The RMS position norm difference of the fingertip compared to an optical system was 5 ib 0.5 mm (mean ib 
standard deviation) for flexion-extension and 1 2.4 ib 3.0 mm for combined flexion-extension abduction-adduction 
movements of the index finger. The difference between index and thumb tips during a pinching movement was 
6.5 lb 2.1 mm. The dynamic range of the sensing system and filter was adequate to reconstruct full 80 degrees 
movements of the index finger performed at 1 1 6 times per minute, which was limited by the range of the gyroscope. 
Finally, the reliability study showed a mean range difference over five subjects of 1 .1 ib 0.4 degrees for a flat hand test 
and 1 .8 ib 0.6 degrees for a plastic mold clenching test, which is smaller than other reported data gloves. 

Conclusion: Compared to existing data gloves, this research showed that inertial and magnetic sensors are of 
interest for ambulatory analysis of the human hand and finger kinematics in terms of static accuracy, dynamic range 
and repeatability. It allows for estimation of multi-degree of freedom joint movements using low-cost sensors. 



Introduction 

Analysis of hand kinematics is important in several appli- 
cation areas, such as rehabilitation, sports, ergonomics 
and animation industry. In particular, ambulatory tracking 
of the whole hand configuration is valuable for kine- 
matic assessment under daily life conditions. This paper 
describes a new kinematic tracking system for the human 
hand which is based on inertial and magnetic sensors and 
offers various benefits compared to existing systems. 

Current hand capturing systems can be divided in two 
categories, namely camera-based systems and datagloves. 

Camera-based systems either use the contours of the 
hand or are guided by markers attached to the finger seg- 
ments. The major drawback of camera based-systems is 
that the measurements to be performed are restricted to 
the volume in which the cameras are placed. In addition. 
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occlusion of the hand-segments or markers result in a 
non-observable situation, inducing a poor estimate of the 
hand pose [1,2]. 

Datagloves form a large group of sensing devices that 
are worn on the hand. They differ in the way kinematic 
information is obtained. Two popular sensing methods are 
resistive-bend sensors and optical fiber sensors, with the 
latter one giving the highest accuracy (< 1 deg), [3]. 

Disadvantages of both methods are related to sensor 
placement. Both measure the relative orientation of artic- 
ulated segments by mounting the sensor across the joint 
of interest. This requires an accurate alignment of sensors 
with the particular joint. Often, re-calibration during util- 
isation is necessary to mitigate estimation errors due to 
sensor displacements. 

A third sensing method used in datagloves is based 
on local magnetic actuation. Those sensors provide a 
high resolution without crossing finger joints. How- 
ever, the cost of such a system rapidly increases as 
the degrees-of-freedom required increases. In addition. 
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a magnetic actuator is required and manipulating ferro- 
magnetic objects could interfere with the actuation sig- 
nals [4,5]. An exception are passive magnetic systems, 
which are low cost and easy to wear [6,7] . However, they 
only allow to estimate a reduced set of kinematic finger 
variables. 

A general disadvantage of datagloves is the lack of user 
customisation for individual subjects' hands and obstruc- 
tion of tactile sensing from the palmar surface of the hand. 
Often this inherently goes with mounting space required 
for embedding the sensors in clothing. 

Inertial and Magnetic Measurement Systems (IMMS), 
containing inertial and magnetic sensors, have proven 
to be accurate in estimating body segment orientations 
without the need for external actuators or cameras [8]. 
The availability of Micro Electrical Mechanical Systems 
(MEMS) technology resulted in tiny and low-cost IMMS 
devices that can be implemented in textile clothing easily 
without impairing the freedom of movement and tactile 
sensation. 

A glove system using accelerometers was presented in 
[9]. The system uses six dual axis accelerometers placed 
on the back of the hand and fingers. It was able to 
detect different static postures of the hand, which is 
useful for sign recognition. An extended version using 
triaxial accelerometers was presented, which was able 
to recognise more complex postures and simple ges- 
tures as well [10]. However heading observation was not 
examined and only a limited number of joints could 
be measured independently. Often, existing glove sys- 
tems have been extended with a single IMMS placed on 
the back of the hand providing 3D orientation of the 
hand. 

A glove instrumented with multiple IMMS' has never 
been proposed to our knowledge. We propose a novel data 
glove that uses inertial combined with magnetic sensors 
placed on various hand and finger segments which is able 
to accurately assess full 3D hand and finger kinematics. 
Multiple extended Kalman filters (EKE) are designed to 
estimate the optimal orientation trajectories of hand and 
fingers. Change in hand position can be measured during 
short movement intervals. 

In addition to presenting the instrumented glove, 
including sensor fusion methods, we evaluate the static 
accuracy, dynamic range and reproducibility of the 
system. 

Methods 

The kinematics of each finger and thumb are treated indi- 
vidually and calculated using forward kinematics outlined 
in the next section. Subsequenly, four sections exploit 
an extended Kalman filter for the calculations of optimal 
relative finger, and absolute hand kinematics. Finally the 
experimental methods will be elucidated. 



Determination of phalangeal joint angles and finger tip 
position 

The articulated finger configuration can be modeled as 
a kinematic chain, originating from the hand coordinate 
frame see Eigure 1. Eor the left hand this frame 
is defined by the j-axis pointing to the metacarpopha- 
langeal (MCP) joint of the middle finger (distal), the ;v-axis 
pointing outwards with respect to the back of the palm 
(dorsal) and the z-sxis is defined according the right- 
handed coordinate frame (radial). The proximal, medial 
and distal phalanges are modeled as rigid bodies of which 
the local coordinate frame is defined such that the 2:-axis is 
aligned with the functional flexion-extension axis (radial) 
of the joint and ;\:-axis pointed dorsally. This definition is 
in accordance with the ISB [11] recommendations with 
positive angles for flexion (2:-axis), abduction (x-^xis) and 
pronation (y-axis). 

The position of the finger tip p^, expressed in the 
hand coordinate frame, see Figure 1, can be derived using 
forward kinematics: 



1 



H 



HD 



1 



(1) 



Where, the transformation between two consecutive bod- 
ies IS expressed by H^^, H^^ and H^^. The superscript 
denotes the two coordinate frames of which the transfor- 
mation is described; Hand (H), Proximal (P), Medial (M) 
and Distal (D). The total transformation H™ is given by 
the product of each consecutive contribution: 



Oj 1 



(2) 



where R(q^^) is the orientation of the distal phalanx with 
respect to the hand, and is the position of the distal 
frame expressed in the hand frame. The rotation matrix is 
defined by a unit quaternion, described in the Appendix, 
because they require a minimal set of parameters and have 
some appealing mathematical properties [12]. 

The relative orientation between two bodies can be 
obtained by solving the following differential equation 
[12]: 



(3) 



where q''^ is the unit quaternion describing the orientation 
of frame with respect to frame O is the quaternion 

multiplication operator [12], and (J- is the angular velocity 
of body y with respect to ^/ expressed in frame ^j. 

The relative angular velocity (J-j is obtained by sub- 
tracting the absolute angular velocities of two articulated 
bodies. The angular velocity of a single body is measured 
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Figure 1 Sagital view of the left index finger. Given are the coordinate frame definitions for liand (^f^), proximal (^p), medial (^m), distal (^d) 
segment, and corresponding joints: Meta Carpal Phalangeal (MCP), Proximal Inter Phalangeal (PIP) and Distal Inter Phalangeal (DIP). To all segments 
a triaxial gyroscope-accelerometer combination was attached. In addition the hand and distal segment include a magnetometer as well. The 
coordinate frame of various sensors is indicated with an S placed in front of the letter that indicates the segment. The position of the finger tip 
expressed in the hand frame can be calculated using the joint positions pjj and relative orientations R'^, were are two connected segments. 
Figure modified from Wu et al. [1 1]. 



using an 3D rate gyroscope, whose output can be 
modeled as: 



Gb 



(4) 



the nominal quaternion q''^ and error quaternion 
given by: 

q^J = q'j o Sq 



(6) 



where o)^^ is the angular velocity of the body with respect 



Subsequently the error quaternion can be approximated 
using helical angles 80^^: 



to a global frame expressed in the body frame, a ^ [l \^0^^^ 



slowly varying sensor bias and independent identically 
distributed (i.i.d.) white Gaussian noise. Subsequently, 
the relative angular velocity between two linked bodies 
(/ and j) can be modeled as: 



(7) 



4 = (>4 - - 4 ) - R^' (^ - b^a - 4) 



(5) 



Filter design: relative finger orientation 

An extended Kalman filter structure is designed for opti- 
mal estimation of phalangeal orientations. The filter oper- 
ates on the error of the actual state. This method has 
an excellent reputation in navigation purposes for air- 
planes and satellites [13] and, more recently, for MEMS 
based IMMS tracking as well [14-17]. It is advanta- 
geous to ordinary extended Kalman filtering because dif- 
ferences in estimated and true orientation is assumed 
to be much smaller than the actual orientation dif- 
ference, which eventually result in a smaller lineariza- 
tion error. In addition, it is an appropriate method 
to circumvent the constraint in orientation descrip- 
tions. We will use the multiplicative error quaternion 
method [13], where the filter operates on the error 
quaternion which can be expressed as a non-constrained 
vector. Parameterization of the true quaternion q^j by 



where 0^^ is the unit vector indicating a rotation axis and 
8 is the magnitude of the rotation around that axis. For 
each finger and thumb a single Kalman filter is deployed 
of which the structure is illustrated in Figure 2. 

The filter uses a general state space model for dynamics 
Xk^i and measurements y^: 



^k+\ =f(^k) + V 
yk = H^k) + e 



(8) 
(9) 



Where f{Xk) and h{Xk)> denote the transition and mea- 
surement function respectively. Process and measurement 
noise contributions are given by v and e. The state x is 
defined as: 



'P qPM qMD 



(10) 



where is the finger tip position, q^^ are the relative ori- 
entations between phalangeal segments. Because MEMS 
based rate gyroscopes have a low bias stability which 
would result in erroneous estimates of the orientation 
when the gyro output is integrated over long periods, gyro 
bias {b^iJ = 1..4) has to be estimated over time, and 
therefore included in the state vector. 
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k = k^l 

Figure 2 Filter topology for each finger. After initialization of botli 
tine state x and error state Sx, a measurement update of error states 
according tine measured acceleration and magnetometer signals iy^, 
y^) and biomechanical dimensionaly information (DIM) can be 
performed. Thereafter states are updated, which are the relative 
orientations, finger tip positions and gyro bias estimates. 
Subsequently gyroscope information y^^ is used as an input to 
perform a update of the process (time update). Finally signals are 
propagated {k = k -\- ]), and the described procedure is repeated. 



During each measurement update step, the Kalman gain 
is calculated and the error state with its covariance are 
updated according [18]: 



Pt = [^-K,H,]P- 



(12) 

(13) 
(14) 



Where, the minus and plus sign denote the a-priori and 
a-posteriori estimate respectively, H denote the linearized 
(or sensitivity) matrix of the measurement equation h{6x)y 
I is an identity matrix, R is the measurement covariance 
corresponding to the measurement uncertainty and 8y the 
difference in estimated and measured sensory input. 

During the state update, nominal states are updated 
accordingly to: 



ql= qlOexpilSO^^) 
bk = bk-\- 8b 



(15) 



where exp denotes the quaternion exponential, given in 
the Appendix. In addition, the tip position is updated 
according to equation 1. 

Finally, during the time update, error states are set to 
zero and the corresponding covariance P, is propagated 
according the discretized process model F (described in 
the next section) and process noise Q. 



Pm = FdPkfJ + Qd 



(16) 



Relative orientation filter: process model 

The estimated angular velocity is given by, see equation 5: 



(17) 



The filter operates on the error state which is defined as: 

8x = [ 80^^ 80^^ 80^^ 8b^^iY (11) 

which include the error angles of various relative orienta- 
tions and the error bias estimates of gyroscopes. 

After state initialisation, the filter is fed with informa- 
tion from various sensors and performs each iteration 
one or multiple measurement updates, a state update 
and time update. We distinguish measurement updates 
from accelerometer and magnetometer, both denoted 
by yyector biomechanical information, denoted 

by 

ybiomech ' 



The true orientation is derived using a rotation matrix 
approximation of equation 6 where the nominal orienta- 
tion is given by the actual estimate: 

R^>'^R^>[l+[5(9^^]^] (18) 

where [] ^ denotes a skew-symmetric matrix. Finally, one 
can deduce the following equations to describe the error 
propagation process (A detailed derivation of the 
equation is given in the Appendix): 

80'^ = -^aJ.:^^80^^ + 8J.J (19) 
8bn = eb (20) 
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using equation 5, 17 and 18 yields the difference angular 
velocity: 



S(o - = (o - — (U-- 

ij ij ij 

and difference in gyro bias with: 
8b = b-b 



(21) 



(22) 



Relative orientation filter: measurement equations 
Acceleration and magnetic field update 

Each phalanx contains a 3D accelerometer that provides 
information of both inclination and experienced inertial 
acceleration. The output of an 3D accelerometer can be 
modeled by: 



(23) 



where a!^ and are the inertial and gravitational accel- 
eration respectively, both expressed in global coordinate 
frame, R^^ is the orientation from global to body coor- 
dinates, is a slowly varying sensor bias and is 
i.d.d. white Gaussian noise. The difference in accelerations 
experienced by two consecutive rigid bodies provides 
information about the relative orientation. In a pseudo 
static situation {a^ ^ 0) the difference in inclination 
can be obtained. During movements the contribution of 
inertial accelerations is not negligible with respect to the 
gravitational acceleration. However, if the rotational accel- 
eration part is significantly smaller than the translational 
part, it is assumed that both bodies undergo the same 
acceleration. In this particular situation the 3D accelerom- 
eter pair might provide both inclination and heading 
information. 

On the hand and finger tips a 3D magnetometer is 
deployed. The output can be modeled (in absence of 
ferromagnetic materials) as 



R^^m^ + em 



(24) 



where is the local static magnetic field, and is i.i.d. 
white Gaussian noise. The magnetometer gives informa- 
tion about the heading difference between various bodies. 

A general equation for both accelerometer and magne- 
tometer vector outputs yy^ctor ^^^^ ^ measurement 
update in our filter. Consider two vectors, and r', which 
are either an accelerometer or magnetometer output, 
measured in frame and respectively. Now the esti- 
mate of given in frame / is determined by the estimated 
orientation R^^: 



The difference between the true and estimated vector 
should be related to the error angle 80 and obtained by 
equation 18 and 25: 

vector = - r' (26) 

Where Cr is i.i.d. white Gaussian noise of the particular 
vector measurement. 

As being mentioned before, two assumptions according 
the accelerometer and magnetometer measurements are 
made: 

1. Only inertial accelerations due to translational 
movements are to be expected 

2. The local static magnetic field is homogeneous 
throughout the whole hand 

Obviously these conditions are easily violated during daily 
life tasks and would therefore deteriorate the kinematic 
estimates. It is therefore necessary that the validity of rele- 
vant signals is tested before being handled by the Kalman 
filter. A decision algorithm is used that either accepts 
or rejects accelerometer or magnetometer measurements 
using the following conditions: 

• The absolute difference in magnitude of both the 
magnetometer and accelerometer output pairs is 
approximately equal: 



(27) 



• Accelerometer information is only accepted if the 
angular acceleration is negligible compared to the 
linear acceleration. This can be tested by the absolute 
angular velocities of both consecutive bodies which 
should be significantly small: 



I {vll 



(28) 



f' = R^>r^' 



(25) 



Where 

€a,b ^re chosen according the desired movement 
complexity. In addition, the measurement covariance of 
can modified each iteration step such that it scales with 
the error of above equations. 

Joint dimensionality constraint 

In addition to sensory input measurements, biomechani- 
cal model constraints imposed by the morphology of the 
hand are enforced by considering the joint constraint as 
an artificial measurement. The limited degrees of freedom 
of phalangeal joints highly constrains the relative orienta- 
tion between segments. In general pronation-supination 
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is not permitted within all phalangeal joints. In addi- 
tion, only the MCP joint allows abduction-adduction. The 
pronation-supination angle (ymcp) can be described as 
the angle between the z-axis of the hand and ;v-axis of 
the proximal finger frame minus 90 degrees, which can be 
modeled as [17]: 

YMCP = ulf^ ' Up^ + ey (29) 

where • is the inner product operator, Cy is i.i.d. white 
Gaussian measurement noise, and u denotes a unit vector 
which is projected onto a second body frame: 

4^ = (13 - [5^^^]^) M„ u, = [0 0lY (30) 

u^^ = R^^ (I3 + [80^^]^) u^, u^ = [lO 0]^ 

Neglecting the error product terms, and assuming that 
Ymcp = 0> yields the estimated error pronation- 
supination angle ((5y ) as function of estimated error angles 

m: 

^ytiomech = YMCP YMCP (31) 

= i^Hz ' ^pj - {^Hz ' ^p.) + 

^[k^^m^u^Y^^^t^x + ey 
-[[u,]^80fR^^R^^u^^ey 

^ (uf,R''''[u^]^-u^¥''[u,]^yO+ey 

The finger s PIP, DIP and thumb s Inter Phalangeal (IP) 
joint constraints can be modeled in a similar fashion, only 
now the abduction-adduction angle P is constrained as 
well: 

Ppip = ' u^Mz + ' yPiP — ' 

Pdip = • + ; ydip = ' + 



Filter design: absolute hand filter 

The absolute hand orientation is calculated in a similar 
fashion as the relative orientations except that the orien- 
tation of the hand is calculated with respect to the global 
navigation frame see Figure 1. 



An additional Kalman filter derived from Roetenberg et 
al, [8] is exploited to estimate the optimal absolute hand 
kinematics. The filter state {\p v 5^]) equations are given 
by: 

= (33) 

where is given by equation 23 after rotation from sen- 
sor frame ^sh to hand frame The position and 
velocity are obtained by strapdown integration [19], 
and (Oq^ is the angular velocity of the hand with respect 
to global frame expressed in hand coordinate frame. 

Orientation drift is prevented by applying a measure- 
ment update using the accelerometer and magnetometer 
pair positioned on the hand, where the sensor output are 
projected onto the gravitational acceleration and earth 
magnetic field respectively. 

Without additional position information, integration 
drifts immediately occurs. Therefore, the position esti- 
mate is only acceptable for a couple of seconds. In order to 
reduce drifting errors, an additional zero velocity detec- 
tor is implemented to detect no-movement situations in 
which the integrators are set to zero [20] . 

Experimental methods 

The instrumentation hardware of the experimental setup 
is depicted in Figure 3. The system contains multiple 
strings of flexible-rigid printed circuit board (PCB) which 
are mounted on the dorsal side of hand, fingers and thumb 




Figure 3 Glove hardware consisting of multiple printed circuit 
board strings which are attached to each finger segment. Every 
segment contains a triaxial gyroscope accelerometer pair wliicli are 
connected using a flexible PCB structure. In addition, the finger tips 
and the back of the hand contain a triaxial magnetometer. 
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using double sided adhesive tape (experiment 1, 2, 3) 
or mounted on an polyamide/elastane fabricated glove 
(Falke) (experiment 4, 5). Each string deploys three triax- 
ial gyroscope and accelerometer pairs (ST LSM330DLC), 
one for each finger or thumb segment. In addition, a tri- 
axial magnetometer (Honeywell HMC5983) is placed on 
the finger s tip and on the back of the hand. Sensor data is 
sampled (@200 Hz for the gyroscope and @100 Hz for the 
accelerometer and magnetometer) by a microcontroller 
(Atmel XMEGA), collected by a master microcontroller 
(Atmel XMEGA) and subsequently transmitted via USB to 
the computer. Online sensor acquisition and filter execu- 
tion is performed using MATLAB . Parameters of process 
and sensor noise distributions is given in Table 1. 

The medical ethics committee acknowledged that med- 
ical ethical approval was not required for all experiments 
described in the next subsections, because the intent of 
this study is assessment of the proposed system and not 
the subject s task performance. 

The following subsections describe, firstly the sensor to 
segment calibration, next, a study with an optical refer- 
ence system with one subject (experiments 1, 2), subse- 
quently, a second study with one subject without optical 
reference (experiments 3, 4), and finally in a third study 
the repeatability of the system which was performed by 
5 subjects. It should be noted that in all experiment the 
subject was seated at a table. 

SensoMo-segment calibration 

Assessment of hand and finger kinematics, and thus all 
described experiments, requires a mapping from sensor 
coordinate frames to the corresponding segment frames. 
In this typical calibration problem we seek for the trans- 
formation matrix from sensor frame to segment body 
frame. First a coordinate frame should be defined within 
the particular segment of the hand, which is given in the 
first methods section. Next, after attachment of the sensor 
PCB to the finger segment, one should determine the ori- 
entation between both frames, for example between ^SH 
and see Figure 1. 

In order to construct this relative orientation, subjects 
were asked to perform the following procedure: 

1. Place the hand on a flat surface with the back of the 
palm pointing up. This defines the 



Table 1 Standard deviation values of accelerometer, 
magnetometer, angular rate, gyro bias and biomechanical 
constraint noise 





ea 


em 


en 


eb 








(m/s2) 


(mG) 


(rad/s) 


(rad/s) 


(deg) 


(deg) 


a 


4*10-2 


10-3 


5 * 10-4 


10-4 


10-5 


10-5 



Given values are determined emperically using allan variances or obtained from 
datasheets. 



abduction-adduction axis (x), given by the 
accelerometer output: Cx^^ = 

2. Raise the hand and flex the MCP, PIP and DIP joint 
of all fingers. During movement avoid abduction and 
adduction of the MCP joint. This defines the 
flexion-extension axis given by the gyroscope output: 



The sensor to segment orientation is given by: 
R^eiSen^^^Seg (.^e, ^ ^Se,^^ ^^^^ 

The phalangeal segment lengths Pij were approximated 
by first palpation of various joints and subsequently mea- 
suring the positions using a ruler. Alternatively or as a first 
guess, segmental lengths can be estimated using a regres- 
sion model of the hand and a measure of hand width and 
length [21,22]. 

Finger tip position comparison relative to an optical system 

In the first experiment the accuracy in terms of finger tip 
position of one subject was analysed. Two tests were per- 
formed, in which the the subject started and finished with 
their hand flat on a table top and repeated each movement 
sequence 10 times with an interval of 1 minute. 

In the first test, the subject was asked to repeatedly flex 
the index finger up to maximum flexion angle of MCP, 
PIP and DIP joint respectively, while having the arm hori- 
zontally stretched such that the back of the hand's palm is 
directed upwards. The cyclical movement has to be per- 
formed five times with a time period of approximately 
1 second. 

In a second test, the subject made circular-like move- 
ments with an stretched index finger while the hand main- 
tained a static posture such that the ab-adduction angle of 
the MCP joint was maximised each repetition. Similar to 
the first test, the cyclical movement was performed five 
times with an period time of approximately one second. 

The estimated finger tip position was compared to the 
measurement output of an optical tracking system VZ- 
4000 (PTI VisualEyez). Both hand and index finger were 
instrumented with active optical markers placed on top 
of the inertial sensors, see Figure 4. An additional marker 
was placed on the index finger's tip. The position of tip is 
expressed in a common hand coordinate frame defined by 
the hand markers and aligned with the hand frame of the 
inertial sensor placed on the hand. 

Index finger and thumb pinching motion 

During the second experiment a subject was asked to per- 
form cyclical pinching movements of the thumb and index 
finger in which the tips of index finger and thumb touched 
each other at the end of the pinching movement. A spot 
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Figure 4 Hand and index finger instrumented witli both inertia! sensors and active optical marlcers. Four markers on the back of the hand 
define the hand coordinate frame. 



was drawn on both tips and the subject was asked to coin- 
cide the spots during pinching as accurate as possible. The 
movement was performed 10 times in which each trial 
contains 10 cyclical pinching motions. The subject kept 
contact during the touching phase for about one second 
in which the hand could be oriented in any direction. Dur- 
ing the touch phase, the position of both index finger and 
thumb tip was estimated. 

Dynamic range 

In order to illustrate the ability to keep track of rapid 
finger movements a test was performed in which 30 
repeated flexion-extension movements with all fingers 
joints were made. The subject was asked to start with 
the hand flat on a tabletop, raise the forearm and per- 
form full hand opening and closing movements according 
a metronome tempo of 116 BPM. This is the highest rate 
that can be achieved before clipping of the gyroscope 
signal (2000 deg/s) occurs. The PIP angle of the index fin- 
ger was evaluated during the repeated flexion-extension 
movement. It was asked to keep the thumb joints extended 
such that the range of the PIP joint is constrained 
by the subjects minimum and maximum PIP flexion 
angle. 

Repeatability 

The reliability of the system was assessed by determin- 
ing the repeatability of the finger joint orientations in a 
defined hand posture. A standardized dataglove evalua- 
tion protocol, proposed by Williams et al [23] and fre- 
quently applied to evaluate datagloves [4,24] was partly 
adopted. The protocol includes a gripped and flat hand 
position test respectively. For both tests Willams dis- 
tinguished between with and without donning-doffing 
between the measurements. The donning-doffing tests 
were excluded in our study as the current hardware is 
not integrated in textile and the non donning-doffing tests 
mimic the proposed applications sufficiently over a short 
and long measurement duration. 



Five healthy male volunteers, aged 21-53 years, with no 
known hand disorders participated in this experiment. 
All tests were performed by the subjects left hand, and 
depending on the size of the hand, either a small or large 
instrumentation set was fastened. The difference between 
a small or large instrumentation set lies only in the length 
of the flexible PCB structure. 

In the first test subjects placed their flat hand on a table 
top within a designated area. This area is indicated by 
the contours of the subjects hand and used as a guide- 
line to securely maintain hand position during the flat 
phase. Then the subject was asked to raise his hand, flex 
all finger and thumb joints, maintain this posture for 6 
seconds and finally return back to the flat phase posi- 
tion for 6 seconds. This flexion/flat cycle was repeated 
10 times. 

The second test comprised a hand posture where a 
plaster mold was clenched. Prior to start of the experi- 
ment the subject moulded a heated thermoplastic material 
(ProtoPlast®) which returned to a solid state upon cooling. 

During the experiment the participant clenched the 
mold for 6 seconds and subsequently released for 6 sec- 
onds. This clench/release cycle was repeated 10 times. 
Subjects were allowed to maintain the hand in any orien- 
tation during the clench phase. 

For both tests the subject was asked to repeat the 
measurement 10 times, where a pause of 1 minute was 
included between each measurement. 

The repeatability of both tests is indicated by the range 
and standard deviation (SD) of all joint angles over all tri- 
als and subjects during the flat phase (test 1) and during 
the clench phase (test 2). 

Results 

Finger tip position comparison 

As mentioned in the Methods section, two movement 
conditions were performed and compared to an opti- 
cal tracking system. Prior to start of the experiments a 
required calibration trial was conducted to firstly align 
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the hand coordinate frames of optical and inertial sensors, 
secondly to obtain the position of the MCP joint expressed 
in the hand coordinate frame and finally to obtain the 
position of the tip LED expressed in the distal coordinate 
frame. 

On the left, Figure 5 shows the position of the finger 
tip with respect to the back of the hand for a represen- 
tative trial of the first experiment where the index finger 
was flexed and extended repeatedly. The error in tip posi- 
tion is defined as the absolute distance difference between 
estimated tip position by our inertial sensor system and 
optically measured tip position. It can be seen that the 
largest error contribution is caused by an error in the 
z-direction during maximum flexion (up to 10 mm). 

On the right. Figure 5 shows a representative trial for 
the second movement type where circle shaped move- 
ments were performed with the index finger. A large error 
is mainly visible in the minima of the z-direction which 
corresponds with the maximum abduction angle of the 
MCP joint. 

Table 2 shows the RMS differences between both mea- 
surement systems of both movement types. Mean and 
standard deviation values over 10 trials is given. The fin- 
ger tip position correspondence was within 5.0 zb 0.5 mm 
for index finger during flexion movements and 12.4 zb 3.0 
mm during circle shaped movements. For both move- 
ment types the largest contribution of the total error was 
due to the difference in the :2-direction. This is presum- 
ably caused by a misalignment of both sensor coordinate 
frames and the accumulation of errors caused by joint 
model imperfections and an incorrect estimation of seg- 
ment lengths. 



Table 2 RMS differences in estimated (inertial) and 
measured (optical) index fingertip position 



Flexion Extension Circles 





Mean (mm) 


std. dev. (mm) 


Mean (mm) 


std. dev. (mm) 


Pd,x 


2.2 


0.4 


5.7 


2.2 


Pd,y 


2.0 


0.5 


4.9 


1.3 


Pd,z 


3.3 


0.6 


9.7 


2.1 


WPdW 


5.0 


0.5 


12.4 


3.0 



Both movement types (Flexion/ Extension and Circle-like) were repeated 10 
times. 



Index finger and thumb pinching motion 

In the third, so called pinching test, the distance difference 
was calculated during each contact phase of the thumb 
and index finger tips. 

In order to minimise reconstruction errors due to the 
rather coarse ball-socket model used for the thumb s CMC 
joint, the subject is instructed to avoid rotations of this 
joint as much as possible. However, this constraint did not 
hinder the ease of movement. The average RMS distance 
difference over all pinched movements is 6.5 ±2.1 mm 
which is 3.7 zb 1.2% of the maximum tip distance differ- 
ence, obtained during the maximum extension phases. 

The average distance displacement of index finger and 
thumb was 65.6 zb 2.2 mm and 15.7 zb 2.5 mm respectively. 

Dynamic range 

A representative trial of a rapid flexion motion of the 
index finger is depicted in Figure 6. Shown are the esti- 
mated PIP angle along with the corresponding angular 
velocities of the index finger. 



-60 





si 140 

a 120 

i 

I 80 




Figure 5 Reconstruction of the index finger position {x,y,z) during a flexion-extension (left) and circle drawing movement (right) 
measured by the optical system (solid) and estimated by the inertial sensors (dashed). The lower plots show the absolute index finger tip 
position difference. 
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As can be seen, the total transition time was approx- 
imately 0.5 s for a full extension-flexion-extension cycle 
corresponding to the 116 BMP rate of the metronome. 
The difference in helical angles between minimum and 
maximum flexion was about 80 degrees, being deter- 
mined in an independed static trial. Please note that 
the range of approximately 80 degrees is reconstructed 
each cycle with the derivative of the angular velocity 
approaching zero before the movement changes direc- 
tions in several of the cycles, while the angular veloc- 
ity stays within the range of the gyrosopce (+/— 2000 
deg/s). The bandwidth of the filter is apparently large 
enough to track these fast movements through the whole 
range. 

Repeatability 

Figure 7 shows two representative reconstructions of one 
subject during the repeatability tests. On the left figure 
the flat hand phase is depicted, whereas on the right the 
flexion phase is depicted. Table 3 shows the average range, 
which is the mean difference between maximum and min- 
imum angle of each trial, and standard deviation of all 
joint angles during the flat hand phase (test 1) and during 
the clenching phase (test 2) for each subject. In addition, 
mean range and standard deviation values over all subjects 
is listed (1.1 =b 0.4 deg) and (1.8 zb 0.6 deg) as well as the 
mean values obtained from other studies with different 
data gloves. 



Discussion 

Small scaled inertial and magnetic sensors combined with 
a biomechanical model of the hand, embodied in an 
Extended Kalman Filter framework, result in a promis- 
ing tool to assess 3D kinematics of the human hand in a 
quantitative manner. 

In addition to existing glove systems which are often 
restricted to measuring a maximum of 2 DoF per 
joint, full 3D angles can be measured. This allows, for 
example, assessment of pronation-supination of various 
MCP joints. 

To our knowledge existing glove systems have only 
been validated in static situations using repeatability tests 
described by Williams et al [23]. Assessment of actual tip 
positions and dynamic range have never been examined 
but are of great importance when the glove is to be used 
for daily-life manipulation tasks and therefore have been 
evaluated as well. 

We have observed that the estimation accuracy strongly 
depends on the sensor to segment calibration procedure. 
In contrast to our approach, most of the existing data 
gloves measure across the joints and therefore give an 
output which is a direct function of the joint s angle. Cal- 
ibrating such gloves might be easier but also tend to be 
more error-prone after long utilisation periods, as the 
sensor s axis should be aligned perfectly with the anthro- 
pomorphic joint axis. 

The current system requires palpation measurements 
to determine phalangeal segment lengths and position of 
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MCP joints centres with respect to a common hand ref- 
erence frame. Accurate determination of functional joint 
position (MCP) and the axis of rotation (PIP, DIP) would 
improve the accuracy and decrease time and effort needed 
for sensor to segment calibration. The current filter can 
be extended such that those parameters are included 

Table 3 Results of repeatability analysis 



Subject Flat hand Custom plaster mold 





Range 
(deg) 


std. dev. 
(deg) 


Range 
(deg) 


std. dev. 
(deg) 


1 


1.4 


0.5 


1.9 


0.6 


2 


0.7 


0.2 


1.9 


0.6 


3 


1.2 


0.4 


1.8 


0.6 


4 


0.9 


0.3 


1.8 


0.6 


5 


1.3 


0.4 


1.7 


0.5 


Mean value 


1.1 


0.4 


1.8 


0.6 


Dataglove 
(Wise etal.[23]) 


4.5 


1.6 


6.5 


2.6 


Humanglove 
(Dipietro etal. [4]) 


3.8 


1.2 


7.5 


2.4 


Shadow monitor 
(Simoneetal. [25]) 


1.5 


0.5 


5.2 


1.6 


WU glove 
(Gentneretal. [26]) 


2.6 


0.9 


6.1 


1.9 



Mean range (difference between maximum and minimum angle during one 
trial) and standard deviation is given for all joint angles of each subject during 
the hand flat phase and during clenching of a custom plaster mold. In addition, 
mean values of this study and mean values from others studies are given. 



and estimated online. In addition, the dominant func- 
tional segment axes are found by performing dedicated 
movements. However, if a certain neuro-muscular patient 
group may not be able to perform such movements, cus- 
tomised sensor to segment calibration procedures may be 
required. 

Likewise, the optical reference system requires a proper 
sensor to segment calibration. Independent segment ori- 
entation measurements require at least three rigidly con- 
nected markers attached to each finger segment. With 
a specified position accuracy of 0.5 mm RMS, markers 
should be separated sufficiently (> 6 cm) to obtain an ori- 
entation accuracy less than 1 degree. Hence, this is almost 
impossible to accomplish for each finger segment due to 
either mounting space, occlusion difficulties or limitation 
of movement freedom. Therefore, we chose to compare 
finger tip positions calculated by both systems. However, 
this approach has a limitation because the error intro- 
duced in either the estimated orientation or measured 
segment length accumulates to an error in the estimated 
finger tip position, which makes localisation of the exact 
error source much more challenging. As the largest uncer- 
tainty is caused by determination of segment origins we 
expect that the primary contribution to the position dif- 
ference is due to misalignments of both measurement 
systems. 

All joints are considered to be perfect ball-socket hinges 
with 3 DoF. Joint dimensionality is soft-constrained by 
adding uncertainty to non-natural rotation axes which 
allows some joint laxity and imperfections. It should be 
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noted that the soft-constrained updates assume that the 
rotational axes of multi DoF joint are intersecting and 
orthogonally directed. However, these assumptions are 
not valid and results in erroneous reconstructions [22]. 
This for example visible in Figure 5, where the reconstruc- 
tion of the index finger position is much worse when the 
ab-adduction angle is maximal. This could be caused by 
disregarded pro-suppination rotations, non-orthononal 
rotation axes, or a translation of the joint's origin. Likewise 
for the thumbs carpo-metacarpal joint (CMC) a perfect 
ball socket joint is considered, which is obviously not 
the case. Mitigating those errors demands improved, and 
more complex, biomechanical joint models [22,27]. Adap- 
tion of such models is the next step to improve the overall 
performance. 

The described method uses six Kalman filters, one 
for each finger and one for the hand. This decoupled 
approach neglects some existing synergies within the 
hand, which means that not all available and relevant 
information is used in the model. However, it keeps the 
filter manageable in terms of computational resources, 
tuning and prevents unwanted coupling between states 
due to modeling errors. Alternatively, one could choose 
for a centralized structure, with one large state vector. 
This approach allows inclusion of synergies and there- 
fore improve the kinematic estimates. However, the state 
will have approximately 100 elements and might require 
alternative, more sophisticated, filtering methods. 

The current filter assumes a perfect homogeneous mag- 
netic field throughout the hand. When interacting with 
ferro-magnetic object this assumption is easily violated. It 
is therefore necessary to extend the filter such that local 
magnetic field disturbances are tracked during operation 
and dont affect the estimated kinematics. Nevertheless 
the magnetic field is only required for observability in long 
static posture periods. 

For this study sensors were both mounted directly on 
the subject's skin as well as on a glove fabricated by an 
elastic textile. Movement artefacts caused by skin and tex- 
tile deformations have not been investigated thoroughly. 
We expect that those artefacts are negligible with respect 
to errors caused by imperfection of the biomechanical 
joint model Moreover, since the applied PCB's are light- 
weighted and have a flexible structure, visual inspection 
and feedback from the subjects confirmed that the sensors 
can be worn unobtrusively without hampering hand and 
finger movements. Nevertheless, a donning-doffing study 
should be performed whenever a suitable textile has been 
selected wherein the sensors can be integrated. 

Our repeatability study showed similar (flat hand test) 
to slightly better (mold test) results compared to exist- 
ing studies of different dataglove systems. It should be 
mentioned that those studies only evaluated the sensor 
variability over time. Simone et al [25], evaluated the 



"Shadow Monitor" which has a simpler approach and 
showed a comparable reliability. However, only a reduced 
set of joint angles could be measured. This is in contrast 
to our approach where all joint angles were estimated and 
evaluated using custom filters. 

Although the repeatability studies are promising, addi- 
tional testing of the absolute accuracy using an inde- 
pended reference system is necessary. These studies 
should include multiple subjects in which all joint angles 
should be evaluated, especially during arm movements in 
which the hand is oriented in different poses. 

It is shown in various studies that the accuracy of orien- 
tation estimates using inertial sensors is higher than joint 
orientation estimation shown in this study [14-16]. Hence, 
the tracking accuracy of hand kinematics is most proba- 
bly not limited by inertial sensor accuracy nor the Kalman 
filter approach. 

Conclusion 

The first results of our sensing system are favourable com- 
pared to existing datagloves. It is able to adequately recon- 
struct finger tip movements (< 13 mm), high dynamic 
range (116 full range finger movements per minute) and 
adequate repeatability (< 2 degrees). This makes the iner- 
tial sensor approach promising, especially, when bearing 
in mind that consumer-grade inertial sensors are get- 
ting smaller and less expensive, whereas the quality vastly 
improves. This study showed the possibilities and chal- 
lenges to be faced when inertial sensing technology is 
applied for kinematic analysis of the human hand and 
fingers. 

Appendix 

Derivation error angle dynamics 

Derivative of the parameterized orientation: 



gives: 



'-\-qOSq 



^qO (o = ^q O (O O 8q -\- q O 8q 

qQ 8q= \qO(0— ^q Q O) Q 8q 

28q = [O 80Y = {8qQ(0-a)Q8q) 
= Q((o)8q — r((b)8q 

0 -((0-0))^ 



(35) 

(36) 

(37) 
(38) 





1 







((O -(b) -[(o-\-(b]^ 

Where Q((o) and r((x)) are the left and right quater- 
nion product operators [12]. Subsequently, by taking the 
second row and neglecting second order terms gives: 

80 = - [a)]^ 80 + 8(0 (39) 
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with: 

8(o = (o — 0) 



(40) 



Now we can calculate the dynamics originating from 
angular velocity differences measure: 



S(o - = (u. — (0-- 

ij ij ij 



(41) 



-[(y.-^y-R''(y.-^L)] 



4 + 



Non stationary gyro bias can be modeled by a random 
walk process: 

bQ = eh (42) 
The error bias process is given by: 

SbQ = bQ - bQ (43) 

Quaternion mathematics 
Rotation matrix 

R(q) = qq^ + qlh + 2qo [q] x + [q]l (44) 
where q and qo are the vector and scalar quaternion part. 

Quaternion exponential 

0 



exp(0) = 



cos 



I2> 



ll^ll 



■sinll^ll 



(45) 
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